Commuting Functions with No Common Fixed Points)

Introduction. Let/and g be continuous functions mapping the unit interval / into itself which commute under functional composition, that is, f(g(x)) = g(f(x)) for all x in /. In 1954 Eldon Dyer asked whether/and g must always have a common fixed point, meaning a point z in / for which f(z) = z=g(z). A. L. Shields posed the same question independently in 1955, as did Lester Dubins in 1956. The problem first appears in the literature in [15] as part of a more general question raised by J. R. Isbell. The purpose of this paper is to answer Dyer's question in the negative by the construction of a pair of commuting functions which have no fixed point in common. The connection between functions commuting and sharing fixed points appears in several areas of analysis. Perhaps the best-known example is the Markov-Kakutani theorem [11, p. 456], which states that a commuting family of continuous linear mappings of a compact convex subset of a linear topological space into itself has a common fixed point. The earliest relevant work on commuting functions was done in the 1920's by J. F. Ritt, who published several papers in which he investigated the algebraic properties of functional composition as a binary operation on the set of rational complex functions. His most important result from the modern standpoint was a characterization of commuting (or permutable) rational functions [19]. He proved that if/and g are commuting polynomials, then, within certain homeomorphisms, either they are iterates of the same function (f=Fn and g = Fm for some F, n, m), both powers of x, or both must be Tchebycheff polynomials (defined by the relationship Fn(cos x) = cos nx). In either case a common fixed point may be shown to exist, so commuting polynomials have a common fixed point. The subject of commuting functions lay largely dormant until a 1951 paper by Block and Thielman [6] presented some new results on families of commuting polynomials and called attention to Ritt's earlier work. Their paper, together with the connection between commutativity and common fixed points found in other areas of mathematics, seems to have been the inspiration for the questions cited above. In the last few years a number of papers have been published on commuting